Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

Opportunistic computation offloading is an effective method to improve the performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. particular, aim design online algorithm maximize processing capability subject long-term queue stability average power constraints. The practical sense that decisions for each frame are made without assumption knowing future realizations random channel conditions arrivals. We formulate problem as multi-stage mixed integer non-linear programming (MINLP) jointly determines binary (each computes either locally or at server) system resource allocation To address coupling different frames, propose novel framework, named LyDROO, combines advantages Lyapunov optimization deep reinforcement learning (DRL). Specifically, LyDROO first applies decouple MINLP into deterministic per-frame subproblems. By doing so, it guarantees satisfy all constraints by solving subproblems much smaller size. Then, integrates model-based model-free DRL solve problems very low computational complexity. Simulation results show various setups, proposed achieves optimal while stabilizing queues system. Besides, induces particularly suitable real-time implementation fast fading environments.